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ABSTRACT 
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these authors attempt to provide information that could serve as a 
basis for an objective consideration of the limitations, 
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discussions covered in the six papers is the question of the use of 
tests as a means of carrying out the assessment goals and the 
implications of having a pilot program run by a private concern such 
as the Carnegie Corporation. The presentation concludes with a 
transcript of the White House Conference assessment debate (July 20, 

1965) , the resolution of the Council of Chief State School Officers 
on measurement and reporting of educational results (November 1965) , 
the ASCD Executive Committee statement of guidelines for national 
assessment of educational outcomes (January 1966), and the AASA 
Resolution concerning national testing and curriculum (February 

1966) • (Author/EA) 
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Foreword 



IMational assessment of educational perform- 
ance is currently one of the most potentially significant and 
actively debated issues in American education. Unfortunately 
some of the recent debate has shed more heat than light on 
the subject. This publication is an attempt to provide enough 
information on and analyses of assessment and related issues 
to serve as a basis for objective consideration of the possibilities, 
implications, and problems posed by national educational assess- 
ment. It is designed to serve both educators and interested lay- 
men. The views presented include those of proponents, oppo- 
nents, and some in between. 

Readers are urged to use this publication as a basis for 
applying reason rather than rhetoric to this important issue. 



says 
FRANCIS 



National Educational 
Assessment: 



KEp'pEL We Badly Need It 



American education today is woefully short of 
the basic information needed to carry forward our many edu- 
cational purposes, to set sound goals, and to work together to 
reach them. 

Tlie U.S. Office of Education, for example, can report on all 
sorts of things about education: how many teachers we have, 
how many school children, how many school buildings, and 
possibly whether the buildings are painted or not. But as yet 
we do not know how much our children really know, the sub- 
jects in which they are strong or weak, the relation between 
income levels and learning, or a host of other matters. 

One of the topics for discussion last summer at the White 
House Conference on Education was a proposal to assess our 
nationwide educational performance in order to acquire the 
information we need. One might have predicted a calm dis- 
course by educators on how, where, and by what means to 
proceed with this assessment. Instead, the discussion turned 
into controversy, and it is not over yet. 

Thus far, educators have become embroiled not in the need 
for assessment but in the evils of testing. Some have said, and 
still sa\, that we have more tests right now than we know what 
to do witii. Others have declared that national testing would 
ultimately force conformity, or worse, upon all our schools. 

The mischievous word here is "testing," and it is this that 
has generated so much heat. A few comments may help lower 
the temperature a bit. 

Personally I am vigorously opposed to any mammoth na- 
tional testing of all children. I do not think it is either justified 

Dr. Keppel, Assistant Secretary for EducaUon, Department of Health, 
Education, and Welfare, ser\'ed as U.S. Commissioner of Education from 
1962 until January 1966; he was formerly dean of the Harvard University 
School of Education, 1948-62. This article is reprinted from the NEA 
Journal, February 1966. 
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U Kc'l';!;'^'""'" ""^^ ""-"V begun 

Because I tellcvo ihal ihc ihcoiy and icclinoloEv of siai(«il 

ST" Cra-roTz-vu: ;^:^raj 

The Carnegie Corporation has already orcani/ed -i com 
inittee headed by Raloh W Tvlnr nr r> a com- 

re^SrlhT^-'-^^^^^^^^^ 

it is honoH'"'°"''''"'^ '".'^ well-executed assessment would 
It IS hoped, serve several important purposes 

I'lrst It would f-ive the nation as a whole a better 
understanding of the strengths and weaknes es of t l e 
American educational system. Thus, it might contribute 

cation of public and private funds, where they ^ire needed 
what they achieve, and decisions affecting ed^c^tion 

5>econd assessment results, especially if coupled with 
auxiliary information on characteristics of the various re 

Uonal ' '""'.'^ '^''-^ '""^^^^y research on educa- 
uonal problems and processes which cannot now be under- 

Quateiv 'Hl''i^'"i'"'P""^ '"'^ '"''"^ procedures are ade- 
quately developed, international comparisons might be 

taken^fn r''"^ private funds have under- 

taken to develop such procedures. They are exploring promSing 



ERIC 



approaches; this spring, they will try some samplings 
experimentally. . 

These assessments efforts are not designed to test mdi- 
vidual students, or individual schools, or individual teachers. 
They am designed to report on regional or nationwide educa- 
tional levels over a period of time. 

I believe that this matter of national assessment deserves 
the full attention of the education profession. We are not now 
reporting satisfactorily to ourselves, to our states, or to the 
nation on where we stand in education, where we are going, 
where we plan to go. ^ * • 

Tlie American people today expect more of Amencan 
education than ever before. At such a time, isn't it clear to 
all of us as educators that what we don't know can hurt us? 
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National Educational 
BANKH Assessment: Will It 
HOFFMANN GlvG Us a Tfug Picture? 



A governmental testing program involving 
large numbers of students would run the danger of ending as a 
lawcd multiple-choice monstrosity. If it did, the results would 
tell educators little that current tests do not. Even worse such 

L^.w'";- ' °f ^PP^"^'-*' o" « Purblind 

method of evaluation that exploits ambiguity, rewards super- 
ficia ity, penalizes depth and originality, frustrates inspired 
teaching, and corrupts education. 

For example, consider this sort of test item: 

Circle the one that docs not belong: 
duck, cat, dog, monkeij 

A Vermont child circled monkey, reasoning logically that 
It was the- one creature that did not live nearby. His teacher 
however, corrupted by the basic fallacy of "objective" tests that' 
there is only one "best" answer, counted the child categorically 
wrong for not having circled duck. 

It is just i.his mind-stunting type of teaching that is fostered 
by mechanized evaluation. Moreover, even the best of the ma- 

A lor!' Tl^ ^'"^ "° '"''^"^ impeccably constructed. 

A 1964 sample CoUege Board SAT question [eliminated in 19651 
bears this out: In an item testing reading comprehension, the 
passage said that certain atmospheric layers are "transparent to" 
the longer radio waves; the wanted answer said that the layers 
do not affect" the waves, which is not at all the same as beinq 
transparent to them. How could eye glasses function if they did 
not affect the light waves to which they are transparent? 

Dr. Hoffmann, Department of Mathematics, Queens Colleee Citv 
Co"l LTr ' °' "'^ Tyranny of T«.S CroweK 

we 1- -'^^P^P^back. 1964). He is a theoretical physicist as 

well as a professor of mathematics. This article is reprinted from the 
NEA Journal, February 1966. 
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At best, educational evaluation is hazardous. To evaluate 
our national effort in education solely on the basis of the warped 
picture presented by mechanized tests would be calamitous. 
Gifted teachers and educational trailblazers would be rightly 
dismayed to have their work federally evaluated by such mis- 
leading methods, just as artisfs would be if their colorful paint- 
ings were to be judged by persons who were color-blind or worse. 

The situation is not hopeless. While keeping in the back- 
ground, officials of the U.S. Office of Education are encouraging 
the Carnegie Corporation to develop a new and ostensibly unoffi- 
cial national evaluative program. In the purely political sense 
the program is brilliantly conceived. It will not tread on the 
toes of any individual, simply because no student, teacher, or 
school will be individually rated. Only a small sampling of 
students will be tested, and none of these students will be sub- 
jected to more than a small sampling of the total evaluative 
procedure. 

Out of this sampling of a sampling could come a vivid pic- 
ture of the state of American education, because the purpose 
is not so much to rate students or schools as to gather evalua- 
tionol data. Furthermore, the sampling, by i^eatly reducing the 
pressure of numbers, would afford a superb opportunity to 
explore quality in depth. 

Will the Carnegie Corporation seize the opportunity to de- 
velop a meaningful system of evaluation? I am a little skeptical, 
for Carnegie has already farmed out details of the task to leading 
test-making organizations that have been long calloused to the 
ambiguities and other defects of machinc-j^adable tests. 

In attempting to develop an instrument which will truly 
evaluate the education of American students, will the test mak- 
ers do some soul-searching? Will they for instance seek to find 
out what harm the current emphasis on machinc-gradable tests 
may have done to students* powers of written expression? They 
could do this easily by giving questions, calling for written re- 
sponses, and comparing them with those of Canadian students, 
who have not been brought up on a steady diet of mechanized 
tests. 

If the Carnegie Corporation s sampling procedure is devel- 
oped with boldness and imagination to probe educational excel- 
lences and evils by a broad range of methods, it will provide 
valuable information about American education — including cur- 
rent evaluative procedures. 
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Certainly national assessment will exercise coercion— let no 
one deceive himself about that. Indeed, the coercion will have 
federal overtones. If wisdom and luck prevail, however, it will 
be a flexible, enlightening cocr Jon amenable to reason rather 
than a dogmatic coercion of the blindly numerical sort exercised 
by such instruments as the TV ratings and IQ scores, 
th.n '^"^ ^^^^Sie program ends up as little more 

han a rehash of current objective tests and interest inventories 
ler MS have none of it. It will do more harm than good 
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RALPH W. 



A Program of 
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TYLER National Assessment 



-"ecause a national asscssnieiu differs so mark- 
edly Irom kinds of educational evaluation in current use, it may 
be helpful to relate assessment to the range of evaluation pur- 
poses and procedures. One common use of evaluation is to 
appraise the achievement of individual students. This is usually 
done with several purposes in mind. It may furnish a further 
incentive lor students to study because they know they will be 
tested. It may be used as one of the factors in promoting stu- 
dents. It provides information that can be used by the student 
and counselor in planning for further education, and it often 
lumishes one of the bases for awarding scholarships. 

A second use of evaluation is to diagnose the learning dif- 
ficulties of an individual student or an entire class to provide 
hiiormation helpful in planning subsequent teaching. A third 
use of evaluation is to appraise the educational effectiveness of 
a c-urriculum or part of a curriculum, of instructional materials 
and procedures, and of administrative and organizational 
arrangements. 

Each of these kinds of evaluation is an essential part of the 
processes of teaching or administration. Teachers and adminis- 
trators are using evaluation of one sort or another as one of their 
normal procedures. The information gained from these apprai- 
sals IS focused upon individual student's efforts, class perform- 
ance, or the effectiveness of the plans, materials, and procedures 
used by the teacher, the school, or the school system. 

THE NEED FOR ASSESSMENT OF EDUCATIONAL PROGRESS 

There is a fourth use of evaluation which is to assess the educa- 

Dr Tyler is Director of the Center for Advanced Study in the Be- 
havioral Sciences, Stanford. California. This paper was presented on 
February 12. 1966. at the Annual Meeting of the American Association of 
bchool Administrators in Atlantic City. New Jersey. 



11 " 



tional progress of larger populations in order to provide the 
of c^url r'f?''^ information to help in the understanding 
of educational problems and needs and to guide in efforts to 
develop sound public policy regaiding education. This type of 
a.sessn,ent is not focused upon individual students, classroo ms 

.bouf h " V""'' '""^'^^^^ information 
about he educational attainments of large numbers of people 

The distinction may be illuminated somewhat by compar- 
ing the situation in education and in the field of health The 
public has information about the incidence of heart diseases 
cancer, and other diseases for different age and occupational 
TuSr geographic regions. This information 

IS useful in developing public understanding of the progress -xnd 
problems in the field of health where greatest effort'ani supp' t 
may be needed^ At the same time, physicians have evaluative 
procedures to diagnose diseases, to appraise the progress Z 

ments .^"^ '° '^^^^'^ 'he effectiveness of treat- 

ments^ The physicians evaluative techniques are devised to 
serve his purposes and the public health assessments are de- 
signed to provide the public with helpful information. One type 
does not take the place of the other. 

This is a rough parallel to the difference in education Hp 

In^hos^ H T'^' '''''' ''^^^^'-^ and admiSators" 
and those needed to gain information helpful for the -uidance 

Of responsible citizens. Heretofore, little attention has be^en gWen 
m educa ion to the assessment problem because the need for 
wide public understanding of educational progress and problems 
was not wiQcly recognized. Now it is. 

Because uiucation has become the servant of all our pur- 
poses, us effectiveness is of general public concern The edu- 

thTrll T ' n"' '■'^^""'^ "^^"y — 'hTn have 
thus far been available, and they must be wisely used to produce 

maximum results. To make these decisions, dependable infor! 

mation about the progress of education is essential; o hemise 

Yet we do' """'''t ^^'^ ^° -'"-'^ oT^^ 

pendable dnTn T ""'"'''''^ comprehensive and de- 

pendable data. We have reports on numbers of schools build- 
ings teachers, and pupils, and about the monies expended but 

re uUs'LeausT' ^"^^^'^^^''^ -formation on education" 
results Because dependable data are not available personal 

sTc; Tuv ""'^ impressions 'areThe 

sources of public opinion, and the schools are frequently attacked 
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and frequently defended without having necessary evidence to 
support either chiim. This situation will be corrected only by a 
careful, consistent effort to obtain valid data to provide sound 
evidence about the progress of American education. 

STEPS IN THE DIRECTION OF VALID ASSESSMENT 

In recognition of this need, Carnegie Corporation of New York, 
n privnte foundation, in 1964 appointed an Exploratory Com- 
mittee on Assessing the Progress of Education. I was asked to 
serve as Chairman. The Committees assignment is to confer 
with teachers, administrators, school board members, and others 
concerned with education to get advice on the way in which 
such a project may be constructively helpful to the schools and 
avoid possible injuries. The Committee is also charged with the 
development and tryout of instruments and procedures for as- 
sessing the progress of education. The Committee has been 
working on these assignments for nearly two years. 

Broad Scope of Assessment 

Tlie discussions with administrators, curriculum specialists, 
teachers, and school board members clearly recommended that 
the initial assessment include more than the 3 IVs and that it 
ultimately cover the range of important educational tasks of 
the modern school. In harmony with this suggestion, instru- 
ments are now being constructed by four leading test develop- 
ment agencies in the fields of reading and the language arts, 
science, mathematics, social studies, citizenship, fine .Tts. and 
vocational education. In subsequent years, other important areas 
will be included. 

Educational Objectives Identified 

Because the purpose of the assessment is to provide helpful in- 
formation about the progress of education that can be under- 
stood and accepted by public-spirited lay citizens, some new 
procedures arc being developed. In each field, scholars, teach- 
ers, and curriculum specialists have formulated statements of 
the objectives which they believe faithfully reflect the contribu- 
tions of that field and which the schools are seriously seeking to 
attain. For each of these major objectives, prototype exercises 
have been constructed which, in the opinion of scholars and 
teachers, give students an opportunity to demonstrate the be- 
havior implied by the objective. These lists of objectives and 
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■ in various Darts of '^ho P^"'''^"^ Public-spirited citizens living 
paneTspcnrtw da ';^^^^^^^^ -»«ges. Each 

-ent aLut7ad?7bj ilTin^ 'r^rofl^i^ -aking a judg- 

somcthing important for people tr^c Jn ol^ 

I would like to have my clSen learnT-S something 

some revisions of the or fn!, ? T^'^ Process resulted in 
eliminations However T °^ objectives and some 

that every objectWe be n^ -^^^^^^^^^ '° '^"^"^'^ 

by scholars (2) accen e! t ' , considered important 

and C3) deemed desl'abfe bvT '^^'T'''' '''^ '""^ ^^^ool. 
help to^limTnate t^ i,?cVm This should 

current tests in which somf ^'^^'J"'^" encountered with 
representing hoddy schohr.h "'''''^'^ ^^^olar as 

as son,ethinVn:fiVtt°tSull";r?;'^^^^ school people 
as being unimportant or technicalTriviL ' ' ''^"'^ 
Populations, Not Individuals. To Be Assessed 

A national assessment to identifv kinHc nr ^ • 

in education, and problemrrnd Hiffl. i, ^ ^""'^ ^'"'"^ made 
very meaningful uSs seoarl ' ^ 

populations withinThT! . ? '"'^asures are obtained for 
selves and thus p esent S ^"^^"8 'hem- 

and differem pXe^ \o tXX" ^"'/^ ^ P^^-- 
that need to be treated .Pnnro T Particular populations 

ahead, but for some tf2 ' ^ ""'^ '^""^^^ '^e years 

geograplnc loStio^Li Tu^al urh."'Y'°?°'^°"°^^^ 
probably be sicnificanT H u differences will 

probability s^rple for ea"h"of'lQ^ ""'''T P'^" '° « 
following subdTvis ons boys a^d Srlf f '''"^'^ '^^^ 

four age groups fnine Vhir,^ ^ ' g^og^^phic regions, 
divisions by urban suburban n'Tf"' '"'^ '^ree 
socioeconomic levels'. classifications, and two 

vidua^^mlrelfposStTxl^ ~ ^"^ 

beyond that of an indivH ? ? . sampling of exercises far 

all It may be tl^. a ^^''^ '''^ P'^""" '^^es it 

somany eLrdst'^h^tlfTw^rrbrt^rr '^'^^'^ 

^ ^^^^Zr-T^^ witETpoi^ 

— - alf th-~ 
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10,000 persons would furnish a sample of 500 for each of the 
n5;<:pcornr.nt exercises, and no one would have given more than 
30 minutes of his time. Assuming that an assessment would be 
made every 3 to 5 years in order to ascertain the kinds of progress 
taking place, it is very unlikely that many of those individuals 
who participated in the earlier assessments would be involved 
in any of the subsequent ones. Hence, from the point of view 
of the child or adult, no serious demand would be made on his 
time. Furthermore, it is unlikely that the children taking exer- 
cises in later years would be drawn from the same classrooms 
as the eadier ones. Therefore, the demands made upon a teacher 
in releasing a child for half-an-hour will be minimal. The assess- 
ment, though costly, should be feasible and involve little or no 
inconvenience to individuals or to schools. 

Since the assessment does not require that all participants 
be in classes, the exercises to be used are not limited to the usual 
test items. Interviews and observational procedures are also to 
be employed to furnish information about interests, habits, and 
practices that have been learned. Because school objectives 
commonly include these areas, it is necessary to see that some 
assessment is made of the levels of attainment. 

Characteristic Achievements To 8c Measured 

The assessment exercises will differ from current achievement 
tests in another important respect. An achievement test seeks 
to measure individual differences among pupils taking the test. 
Hence, the items of the test are concentrated on those which 
differentiate among the children. Exercises which all or nearly 
all can do. as well as those which only a very few can do, are 
eliminated because these do not give much discrimination. But, 
for the purposes of assessing the progress of education, we need 
to know what all or almost all of the children are learning, and 
what the most advanced are learning, as well as what is being 
learned by the middle or "average" children. To get exercises of 
this sort will be a new venture for test constructors. They are 
required to develop exercises at each age level in which approxi- 
mately one-third represent achievements characteristic of most 
of those at that age level, one-third represent achievements 
characteristic of about half of those at that age level, and one- 
third represent the achievements characteristic of the most ad- 
vanced, that is. the top 10 percent, of that age level. 



l-or 11,0 sample ol' scvcntccn-ycar-old boys ol higher sorin 
ccono„.c status fVo„, nual and s„,all c o he" 

Midwest region, it was lound that: 

fhc/foZ!'.^:'';' '''''''' p--«-ph like 

/erf could write an acceptable letter ordering scvcril 

'tcnis Irom a store like the following 

52/0 took a responsible part in working with other 

employ" ^'"'^ 
Periodic Reports Planned 

FEARS ALUYED 

LtrrtoTd^'t^^3^u:e7^•^^ 

tcstmg program ,nrer I V'''"'"'''^^' '"^^'^'^ual 

those Who make' tl ^o, I on°T°" f ''^'''''"^ 

a school influence he dTc^^^^^^^^^^^ ""'^ '^^'^ in 

and teacher, n , amount of effort of pupils 

local TdSon "pr,ra::i'?" ™' 

from their work TM. • r •' . ^ teachers are deflected 

show nu. No studnnt will f.,L.« , ^^^^iiei can make a 
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will not be assessed at any later time and can gain no desired 
end, like admission to college or a scholarship. 

A second I'car is that such an assessment enables the fed- 
eral government to control the curriculum. This is also a mis- . 
understanding. The objectives to be assessed arc those which 
arc accepted by teachers and curriculum specialists as goals 
toward which ihcy work. They have been reviewed by lay lead- 
ers throughout the country so as to include only aims deemed 
important by public-spirited citizens, This project will report on 
the extent to which children, youth, and adults arc learning 
things considered important by both professional school people 
and the informed public. 

A third fear is sometimes raised that this project would 
stultify the cuniculum by not allowing changes over the years in 
instructional methods and educational goals. It should be made 
clear that the project will assess what children, youth, and adults 
have learned, not how they have learned it. Hence, the assess- 
ment is not dependent upon any particular instructional 
methods. Tor example, we shall report the percentage of 13.-year- 
olds who can comprehend the plain sense of a typical newspaper 
paragraph. We will not be reporting the methods of reading 
instruction that are used in various schools. Or. as another 
illustration, we shall report on the percentage of adults who par- 
ticipate regularly in civic affairs but not on the methods used m 
teaching high school civics. 

The matter of changing educational goals is a relevant 
question because the objectives determine what will be assessed. 
Our plan calls for a review one year in advance of each assess- 
ment of the objectives of each field in order to identify changes 
and to include the new objectives in the next assessment. 

A CONSTRUCTIVF. CONTRIBUTION 

Through the various conferences with school people and inter- 
ested laymen, the Committee has been able to identify concerns 
and problems that such an assessment must deal with. As the 
plans are shaping up, it appears to be possible to conduct the 
project in a way that will not injure our schools but will provide 
greatly needed information. 

The need for data on progress has been recognized in other 
soheres of American life. During the depression, the lack of 
dependable information about the progress of the economy was 
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a serious handicap in focusing efforts and in assessing them 
Out of this need grew an index of production, the Gross National 
iroduct. which has been of great value in guiding economic de- 
velopment. Correspondingly, the Consumer Price Index was 
constructed as a useful measure of the changes in cost of livinc 
and inflation. Mortality and morbidity indices are important 
bases for indicating needed public health measures. Facing the 
need for massive efforts to extend and improve education the 
demand for vaMd information to support the requests and to 
guide the allocation of resources must be met. The assessment 
of the progress of education should make an important and con- 
structive contribution to this purpose. 
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Panel Discussion 
on a Program of 
National Assessment 



This discussion followed Ralph Tyler's address 
at the AASA Annual Meeting on February 12, 1966. The panel* 
was moderated by Martin Essex, Superintendent of Schools, 
Akron, Ohio. George B. Brain, President of AASA, presided at 
the general session and made brief remarks at its conclusion. 

BROWNELL: There is no aspect of public service .in the 
United States which is assessed more critically and frequently 
than American education. 

The pupils assess their schooUng every day, and the assess- 
ment may not be sophisticated, but it is candid, and often re- 
vealing and effective in pointing out the needs and deficiencies 
of the particular school situation; and the parents and the citi- 
zens generally and the press and the writers make their assess- 
ments daily, so that those of us who are in the operating end 
of the school systems are quite aware that there is a national 
assessment of education, and that the ones who are most critical 
are often the ones that get the most notoriety. 

One of the problems is that those are usually assessments 
of smaU pieces of the school program. It seems to me that the 
idea of an operation which reports to the public where we are 
nationally in the attainment of the educational objectives is a 
highly desirable operation, but I sense what is of great concern 
is whether the procedures that are proposed and under way with 
the commendable objective of national assessment of education 
are going to accomplish the genuine national assessment of 
education. 



* Other panel members were S. M. Brownell, Superintendent of 
Schools Detroit, Michigan; Leon P. Minear, State Superintendent of Pub- 
lic Instruction, Salem. Oregon; L. B. Newcomer, Superintendent, Clark 
County Schools, Las Vegas, Nevada; and Harold Spears, Superintendent 
of Schools, San Francisco, California. 
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Darius Green and others had the laudable objective of soar- 
ing in space. That is what they wanted to do. Their procedures 
didn't get the results they expected, and were disastrous for them 
in getting support for what they were trying to do. And many 
educators arc concerned, I think, today, lest the proposeu na- 
tional assessment program result in injuries to education rather 
than accomplishing the stated purpose. It is not that they arc 
opposed to the objectives. I am glad, however, that some persons 
persist in their efforts to fly. despite the tragic results of Darius 
Green s experiments; and I hope the careful and critical examina- 
tion and present plans for national education assessment will so 
direct and modify the plans of those who are pressing for a trail 
that unfortunate results can be minimized and constructive 
results maximized. 

Now, let me turn to some of the questions which I assume 
and hope are being considered by those who are working on what 
is called the national assessment program. One of the persistent 
handicaps in the local school assessment of the status and effec- 
tiveness of the school program is the difficulty of separating out 
and measuring what the school has contributed to educational 
results. 

Let me illustrate. Schools, by law in most states, are re- 
quired to teach the harmful effects of tobacco and alcohol. You 
can figure out the amount of time that schools are able to devote 
to this requirement, and the money that they are able to spend on 
it, compared with the exposure that pupils get to "education" by 
other forms, such as advertising, to offset the school education 
in this one objective. I think many people are wondering how, in 
this assessment program, ;t is proposed to assess the efTective- 
ness of schools on an objecUve which is clearly set forth as an 
objec ive of the schools, but which, at one and the same time, 
may be offset or crippled by the other ^^educational agencies*' 
tha* are at work. 

1 would use another illustration which has to do with good 
hrinan relationships. 1 know of no school system that doesn*t 
< jnsider as one of its major objectifies, or no group of citizens 
Jiat doesn't say one of the major objectives of the schools is, good 
human relationships; and 1 suspect that most of those who are 
concerned about the assessment program are wondering how it 
is going to be possible to evaluate school education effectiveness 
in this area. What is to be the basis: by the relationship of pupils 
to each other in school or their human relationships in the com- 
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munity. by their knowledge of good human relationships or their 
performance at school age or as adults? 

My point is this: I think that many who have been strug- 
gling to assess the effectiveness of schools locally in terms of the 
goals of education that arc subscribed to by citizens and scholars 
wonder whether the national assessment program has or will 
have criteria which can identify the contributions of schools to 
what they report as a national assessment of education 

And many educators arc so much aware of the limitations 
of papci-and-pcncil tests, as Ralph pointed out. in evaluating the 
total school output without relating these results to input that 
they question the value of placing the national investment and 
pressure approval behind a procedure that they assume will be 
merely an extension of present questionable testing practices 

Secondly, I would make one or two other points . Allusions 
were made in your report to the usefulness of the development 
of a gross national product in guiding economic development and 
the consumer price index as a useful measure of the chances in 
cost of living and inflation. 

Now, personally I consider this quite different from the 
value judgments that are implied in an assessment process If 
the objective of the assessment group is to develop an index 
of gross national educational product, rather than a national 
assessment of education, it would certainly clarify to me the ob- 
jective or purpose of the effort in this field. I think that most edu- 
cators would be glad to have developed a gross national education 
product mdex as a tool to emphasize national growth in educa- 
tional productivity. 

If emphasis were placed upon the relationship of expendi- 
ture for schools, on personnel-per-thousand pupils, on length of 
total time spent in school or frequency of change from school to 
school, on the increase in school holding power, the job success 
ot pupils, and so on. as well as the test scores, then I think some 
of those who are worried about a national assessment would have 
their fears reduced. 

Then there are many thoughtful persons, among whom arc 
some of the most outspoken critics of the present conditions of 
schools and colleges, who wonder what are the safeguards that 
are being planned or erected to reduce the educational ties if 
the national assessment program misfires in its intended 
interpretations. 

In any social experiment which could affect importantly a 
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major social process for good or for ill T a«.„T,o .u.. .u 
charge .™u., „,aks carsfuf plans r.afsguardTs ^d ,*"^^^^ 

are responsible for the operations of social progress as to thn^P 
procedures that will protect workers on theVoTect or inni^en 
bystanders agamst a misfiring or a miscalculation of the resuUs 
of ' ""V"""' °^ I that are in the Jnds 

hpff ^l-^' ^ "'''"''^ '° ^^"^ 'hese very pertinent questions 
SffFprp ^'^^"^ 'his assLment p?oces the 

differences that the school makes and other educationTforces 

heZpr''""^?'"'"' ^alogy St ween 

^vin? '"'^^ product or co^ of 

h7r»; H ''^''''^ '° ""'"^ judgments to be made here and 
thirdly, he wants to know whether you can retrieve thk i;c?^, 

^^iSnir-''' ^ cas?;: s^ To b:^'^- 

TYLER: I think these points or questions that Dr. Brownell hnc 

as often mamtained, that in the rural areas their people don't 
read as well, or in the rural areas that you have less adeouate or 
more adequate development of ciUzenship or other thtgs 

The purpose of our assessment, the purpose of usL .^p 
groups, not grade groups, and not schools, in Detroit fo^ex 
ample IS that in taking all the children in a block befo ; ge tine 

n^tlV"" ''"r""" d° «bou prob ems wi 

need to know what our problems are. The children that wouTd 
be sajnpled for the whole Midwest, and not simply for dZu 

s^ook nV°fi" P"^"' ''''''''' P^°^hial schools, p fvate 
schools of other sorts, and perhaps, because we found t h s 
some samples-a larger percentage than we realized-some ch " 
dren who weren't in school at all. 
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Let me then briefly summarize by saying that the purpose, 
first, is to get information about what our national problems are, 
and we are getting them from various sources that are inade- 
quate. There was a book that came out 15 years ago, called 
Wlnj Johnny Can't Read, There was no evidence as to how many 
Johnnies there were who couldn't read. We need information 
of a general sort about how well educated our people are and 
where our problems lie. I realize that at some point in Detroit, 
for example, there may need to be a further analysis in different 
school districts as to what their problems may be and what they 
can do about it; but right now, we have a great deal of money 
being spent— public money— on the ground that there are cer- 
tain groups that are not well educated. We don't really have any 
adequate data on it. We are calking about a prior step, which asks 
just how are our young people coming on and does not try at this 
point to make a diagnosis of where and how it might be improved. 

Again, let's make the medical analogy. We are trying to find 
out just where the incidence of disease is. I want to know whether 
we have diseases and where they are. 

MINEAR: Ralph, I believe that many of the people here have 
heard for the first time the actual project which you and your 
friends are undertaking, and I believe you have converted many 
of us to some new ways of thinking. But probably you have 
aroused some opposition in some other quarters. 

I want to add and say to you that the Council of Chief State 
School Officers, the state superintendents, the commissioners of 
education of the 50 states, went on record in Honolulu a few 
months ago substantially approving a form of assessment; and, 
of course, they went on and said, if and but and so forth. I 
think you have covered those angles. 

I am certain you realize that AASA has a resolution that is 
before the various committees at the present time, opposing a 
form of national testing, and you have assured us on that par- 
ticular point. 

In our state of Oregon, I surveyed the superintendents and 
find that 60 percent of the superintendents who responded to a 
survey that I made were mildly in favor of some type of assess- 
ment, but 40 percent were vehemently opposed. 

Obviously, then, it becomes a highly debatable issue, and I 
have two questions for you. The first is this. On page 147 of 




and I quote: ^ " ^""^''^^'^ ^'PP^ars a statement. 

'"cnt dat.,o„ a Lfform ' L'^^^^^^^^^ achieve- 
assessing the ull rof'edu ^C^^^^^^ P-P°- 
to be eolleeted as a supprement to^l,e ''"'"'^'''^ °^ '^''''^ 
survey and to initiate a su vTv of a f Population 
ing for employrm ^ ^ '''"d train- 

in s " °^ ^^-^ -^"^on 

planned for a new proSL of .n, ?" °^ apparently 
n-nt data on a natioSrbasis Mv r ' '''^h''^^- 
there a relationship between ^ ^""'^ ^« '^is: Is 

Presidents budge in S s p"t ?.l''T'"'-'"' ^''^''"^ the 
are aware? Paiticular statement in so far as you 

there'lieXe^ran'Ll'""^ ' '''' '^'^ 
mounted. You ment oned fht r^''?'"^'^"' P^e^'-*"^ be 

guishedeitizensS eTby ^^/^^^^^^^^^ of distin- 

tional way would be a Presiden^inl n ' '"'^ -^°f°«h. One addi- 
direet allocation to the Un ted ! citizens. The 

responsibility wou^ be a third vvav T °' 'his 
that is to leave it wLe it starr?" • ^'^'^ « fourth way, and 
educational enterprrse ' " ^""'''''^ ^'^^'or of the 

nation Z;;th:T4^^^^^^^ in Z '^oT ''' '' ^'^'^^ °^ 
oral development of^pub^c education ^^'^ S'^"" 

vvelfare clause and poLbly porUon of th^^^^^ "l^" f general 
the Constitution do we hriJl^ I ^ ^''""^ Amendment of 
government. ^ ^aucation into the federal sector of 

assessment underTvay CufdTou If " '° S'^' ^^is 

might be, in turn, transferred ITn T u 'PP'^P"^'^ that this 
of the Compact o S a^esMt °^ ^e first responsibilities 

is inaugurated, ther wHl be demand for'^ '!!'^ ^°"^P^^' 
some job for the Compact of S^^^. 1 i '"'^sponsibility, or for 

this would be an approp ia e^?n',;e?o"f •?? 

agency? ^ transfer of responsibility to that 
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TYLER: The two questions that Superintendent Minear has 
raised arc both questions that are highly hypothetical; namely, 
the possible connection later that the federal i;overnnient mi^lU 
have with this. Actually. 1 don't see any necessary connection. 
Our work has been entirely in the private' sector. We were estab- 
lished by the Carnegie Corporation. Our membership includes 
people who are interested primarily in this question: how can we 
develop a process of assessment that will give the public inrornia- 
tion that wmII not have any harm on the schools and will not be 
an assessment of the schools. 

This emphasis. lor e.<ample. on no smaller geographical 
region than the regions represented by the four in the United 
Slates— Northeast. Southeast. West and Far West— is one means 
of ensuring that we are not talking about comparing one state 
with another. We are not talking about comparing one kind of 
community with another. My own belief is that whatever may 
be the need and the desires ol* the Congress in trying to assess 
their responsibilities, our concern is with the assessment of our 
educational development as a whole, which includes children 
who may have been educated in parochial schools, private 
schools, as well as public .schools, who may have been at homo 
or out of school altogether. 

The question is often raised now: Is it true that such a large 
percentage of our youngsters are unable to pass the test involved 
in getting into the Armed Forces? I am sure that if we took a 
comprehensive sample, we would discover that young people are. 
as a whole, much better able to read and carry on these opera- 
tions than the data we are getting from the draft. We are inter- 
ested in getting balanced, adequate pictures of how well 
educated our people are and where our problems lie. rather than 
the questions which are of concern to you as administrators: 
what can we do in our own school systems to continue and im- 
prove the quality and the quantity of education. 

ESSEX: Ralpli, seated out in front of you are some thrc»e or four 
thousand school superintendents who function in that peculiar 
organism known as the American community. Now. you state 
that you propose to do this in four sections, with two elements 
of the population, various ages, sex, and so on. 

What if Madame X of the PTA, League of Women Voters, or 
a hundred and one other organizations, wants to know of the 
Board of Education or the Superintendent of Schools how the 
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youngsters in Ox-Bow arc doing on tlieso-will this be a public- 
instrument? Is it available? 



TYLER: It will not be a public instrument because the whole 
basis of the sampling depends on having large enough popula- 
tions, so that no one city would have any value in using this 
instrument. It will not be a public instrument It is copyrighted 
by the Carnegie Corporation, so as to retain it entirely for use 
of this sort. 

NEWCOMER: Let me put my questions to Dr. Tyler this way. 
There have been many concerns expressed about the national as- 
sessment program, and usually you hear people talk about the 
national tesUng program— the ills of national testing. It seems 
to mc that in your comments today you have very well attempted 
to show that this is not a testing program, and that assessment 
IS different from the testing. However, I seem to detect a gradual 
change in the communications regarding the assessment pro- 
gram over the past year or several years. We don't see the word 
"comparison" any more. We did see the word "comparison." I 
think, when we first read about it, and I suspect— I want to ask 
the question, whether the controversy that has been going on 
somewhat along this line has had a tendency to narrow the scope 
or to sharpen the process. That is one question. 

The other one is rather dear to me in that it has been ex- 
pressed in several ways before. I think that one of the problems 
we face today is the tendency to equate education with public 
schooling. When you talk about the assessment of education— 
If this IS going to be used to judge the public school— I don't 
know how you are going to delineate this process because I think 
that there are other educational institutions in our society that 
play an even more important role than the public school. If the 
national assessment program is successful, there may be— and 
I ask this in the form of a question: Could there be a tendency 
for the people to quit the great debate in their local communities, 
quit trying to determine what the unique task of the school iri 
our communities is and consider that somebody up there is de- 
termining what it is, and that it is the same for everybody? 

I have a sneaking suspicion that the task of the school in 
Las Vegas is very different than the task of the school in New- 
port Mesa. I wonder if this national assessment program might 
not have a negative effect on local people going into the con- 
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trovcrsy and asking, as iliey sliould, what are the tasks here as 
opposed to other educational institutions. 

TYLER: It is always difTicult to deal with things that have 
not happened. What we can do is consider what we know about 
our own communities and what we have noticed in other fields. 
I have noticed that, for example, having national data about the 
problems of executives in getting heart disease serves in our area 
to make for a special study of the incidence of heart disease in 
our county and helps to focus the emphasis of the Heart Asso- 
ciation upon it. 

I do not understand why having knowledge (if this should 
turn out) that children from low-economic-level groups keep 
getting wider apart from others between 9 and 17, if this turns 
out to be true in each of the regions, why this doesn't cause us 
in Las Vegas or Palo Alto or anywhere else to begin to say: if 
this is a national problem, let's see what our own problems are 
and whether we ought to give attention to it. The business of 
getting more dependable information is not something we have 
to be afraid of. The role of those of us who are administrators 
is to make use of dependable information, to help remind our 
citizens of problems we have, of things we need to do. I don't 
fear more information; I fear either lack of information or the 
dependence upon very biased material. There are many organi- 
zations, as Sam Brownell has pointed out, that are continually 
making an assessment, trying to assess education, and I believe 
that the information about regions, about these divisions that 
we talk about, 192, may not be the things that will be important 
10 years hence. We may find some other factors that we will 
want to look at in relation to these, either because we discover 
we don't have some of these problems or we discover we are 
doing very well with them. By having information of that sort 
of background, to indicate where our regional or national prob- 
lems lie, gives us as school administrators a much better basis for 
saying where we need to put attention in our own communities. 

ESSEX: We seem to be saying here that we are going to keep 
this tool separate from the teacher. We are not going to let the 
teacher get to this instrument because we fear that this would 
direct his teaching toward certain things or cause him to elim- 
inate other elements of his teaching that were not deemed to 
be important here. 
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TYLER: \Vc didn't really say that. 



nosis. They a.e'ul^^dX r " t" ''^^ 
to keep things away f rom the tcS er A ,?>^., Z'' 

-systems in the Lrnfa^Xaio? " ''"'^ 

or some other particular rornnr.M [ ' ^"'"'^ Corporation 
and make up ifs o n ^udv o'^^^ho " n'' '° "^'^^ "^^^^ 
the background o^^he national r " P™'^"'^'"^ '-'S'-'i'^st 
produeti^yor are we not " '""'^'^'""^ '""'^-^ing our 

instrumLt^^h^^aTjrSe''?""^^'^^'' '^^^'^^ 
ments to f.nd out their sUu t"n 8^1^"'" ^""'l 
public has so many rnisconc nl u "^"""""^ '^e 
problems are. bec7usT ve ha'^T^^^^^^^^^^ our educational 
that the Public Health Service provfdes fht ''^'^'''T^^ '^'"g 
from the physicians' reports ' ' ''^^''^^''^ 
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SPEARS: I noticed that even- time. Ralph, that the word 'assess- 
ment" came in. tiicTe was a rumble. Whether vou said it or 
whether Sam said it. Also. I noticed when the word "national- 
came in. there was a l)i{;ger rumble. 

I am not going to ask Ralph any questions, because super- 
mtcndents are not accustomed to ask questions. We are on the 
receiving end of so many of them, wc are out of practice But 
you can imagine wliy we are skeptical, if we arc. in this area 
and why we need to have people such as you analvze it and give 
us the facts as to what was intended, because there is so much 
national management coming in that when anvthing else comes 
on a national basis today, we are bound to he a little skeptical 
until It is proven that it is something that we need and can use 

Even though you say that it is a major change in testing 
programs. I analyze common features, and what we have here 
seems to me to be common with what we have been doing with 
testing since the Army test came out. 

This program has standardized tests, as the other programs 
we have of measurement, and assessment, and it is bound to 
cover subject areas in which you can make tests; and unless 
these test-makers are smarter than the same firms that have 
been making tests in the past. I doubt if they come up with good 
tests m the other areas outside of the academic and skilled areas 

Again they are testing individuals. We arc not going to 
measure individuals, but we will measure school systems- but 
the only way you can get to the school system is to get to the 
individuals and compile scores of numbers of individuals even 
though each takes a partial part of this test. Then you combine 
them and then wc get an assessment on differem areas of the 
country. 

I am a little sensitive to this on the ground that California 
has a state testing program, as a lot of people know, and it is 
a required testing program that we have to go into. It has been 
in effect only two or three years. Wliat they will do with those 
scores, we have no idea. All that we arc concerned about was 
that nobody knew about them, and so we couldn't compare them 

When you say this is not a public instrument, wc people 
who work in public education know that everything that is in our 
office or files today seems to be public. Tliere is no way you can 
get away from it. We arc being badgered all the time for having 
facts that we are trying to hide from somebody, as though we are 
trying to keep something away that the public needs, and we are 



29 



29 



skeptical here again on the national testing program, because it 
will compare districts. 

Maybe it is moving a little too fast for us, I don't know. 
Maybe we don't move fast enough as educators or superintend- 
ents. It is hard for us. We do have our ways of working. We 
have been accustomed to local control and local management. 
When these things come upon us, we are bound to ask for a little 
time. 

On the matter of objectives, we feel that sometimes the ob- 
jectives of education are set by the instruments themselves, and 
if the instruments are limited, the objectives of education become 
what the instruments measured. If we are in that type of move- 
ment, we could be concerned and ask your group to put out a 
little more information about it. 

Getting back to the use of the test, I will quote from Dr. 
Gardner when he was asked to make a further statement on this 
subject and he went into the matter of the use of it. He said: 
"Thus it might contribute to a much more active guide than we 
currently possess to the allocation of public and private funds." 

That bothered me a little because teachers who are in- 
volved in testing programs for school systems sometimes test 
for them. 

There are two features unique to the proposed national 
testing program. One is the coverage of the nation rather than a 
state or local school system or district. The other is the an- 
nounced use of the result as a guide in the allocation of school 
funds. 

If the latter were carried out, local boards of education 
would be confronted with mixed emotions. First, the anxiety 
that their students would not rank high enough. And second, 
that they would rank too high to qualify the district for federal 
funds. 

We have to get that cleared up a little. Are we working 
now? If federal funds will be distributed to the lower-ranking 
districts, we had better tell the teachers to ease off a little with 
the children when they are taking these tests. 

I am not quite sure of that. Take the California situation. 
Three weeks ago these test scores were released. Our school 
system got a beating in the local paper because the headlines 
came out that we were below average, and what was our average 
on that teit compared with these other 143 different districts 
in the metropolitan areas? We ranked 199.5 as compared to 



30 




200 as the average. We were below average! We were 0.5 off 
the beat on that. 1 iricd to answer and get into the papers, which 
we did, and talked from the standpoint of achievement of young- 
sters and whether you measure school systems on that basis, 
because you have to know whom you arc dealing with, to begin 
with, how far you raised them from, and where you had them 
to begin with. So the achievement score doesn't mean much 
unless you go back to the intelligence quotient programs. I indi- 
cated that our school system has a number of culturally handi- 
capped students in the school system, and we may be doing 
better than others that have a higher score. Then I was accused 
of being disrespectful of minority children, and somebody sug- 
gested that 1 wasn't competent to be in that position. 

TYLER: I think Harold Spears has pointed out perhaps one of 
the most important reasons for a great deal of emotional concern 
about any sort of assessment. Yet he has illustrated the fact that 
without tiiis sort of program going on, there is a great deal of 
effort now being made by people to assess unwisely and inade- 
quately. 1 do not believe you are going to stop the concern of 
the public generally for knowing how our schools arc doing, 
because it has reached the point where for ever>' child to get 
anywhere in the United States or any industrialized nation, he 
has to have an education; and because education is so important, 
people care about it, and as long as people care about it, they 
want some sort of appraisal. 

I would rather have an appraisal that has comprehensive 
validity, that does not pretend to appraise a particular school 
system, that looks at our total educational problems. People are 
then able to see where we are coming along, where we don't need 
to worry, where we need to get more attention. This is what we 
are concerned with, because we believe that appraisals — and I 
repeat what both the Superintendent of Detroit and the Superin- 
tendent of San Francisco have said since they are under con- 
tinuing appraisals — we would like to get a more comprehensive 
and adequate base. 

Now, the question that was raised about how this differs 
from standardized tests, I can only refer you back to the paper 
to point out some of the technical differences, but the most im- 
portant one perhaps is the standard job to be done, not a score 
that is determined on the basis of the mean of the group, so that 
the problem of individual testing where you are trying to find 
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individual difTcrcnccs is precisely the problem that has been 
pointed out by Harold in San Francisco. If you arc at the 49ih 
percentile, that is lower than the 50th perceniile. IT you really 
talked about the thini;s that kids can do. the public could much 
more easily see the difl'ercnce between, and how small the difl'er- 
cnce is between. 82 percent who can do so and so and 81 percent, 
than they can the cliflerence between above and below averai;e. 

The reason for concerning ourselves again with regions 
rather than uith any state or school system is that the public 
needs to know where, as a whole, in our areas various problems 
lie. It is then the responsibility of the constituted school officials 
with this a.s a background to look at their own situations, to call 
for help, or to comment on the progress that. is being made. I 
hope, therelore. that we can look at this assessment in terms ol 
a way of trying to provide information that docs not point a 
finger at any school, any person, or any teacher, but does help the 
public to get knowledge of where our problems lie. because we 
do need more and more resources to deal with thom. 

BKAIN: There was a consensus represented in the discussion 
today. That consensus 1 would express as being the imperative 
of improving the quality of education. 
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district does not wish to cooperate, it will be necessary to arrange 
for these exercises to be administered in the home situation. 

Si^ytrt^e il ^iT tabulated and who would 

DR. TYLER: The results will be tabulated by exercises for each 
population. Smce the analysis and reporting would be done by 
official commission-probably one selected by several of the 
foundations so as to be free from political bias or governmental 
responsibihty-it is Ukely that the commission woi^d get a com- 
petent staff to write the report, and the analysis in general would 
deal with particular a.,- as or particular kinds of objectives and 
comment on the progress being made and the problems being 
encountered about which the American people would need to 
give support and help to their school districts in attacking. 

QUESTION: To whom will a final report be made and what use 
do you contemplate will be made of it? 

DR. TYLER: The final report of each assessment would be made 
to the interested lay public, and its use. I think, would be very 
much like the use made of the report of the American Youth 
Comniission in such books as How Fare American Youth which 
alerted the public to the problems of youth in the depression and 
helped to get support for improvements in curricula and in job 
opportunities and other needs of youth. 
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uAon. n . Recipe for Control 

HAROLD C. _ \^ 
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I have designed this paper around a priijciple 
which states two conditions which any national achievement 
testing program whose results are used to compare public school 
systems must satisfy in order to comply with the requirements 
of the pnnciple of state and local control of the pubUc school 
curriculum and the principle of government by consent. I shall 
apply this principle to the national achievement testing aspect of 
NAEP (National Assessment of Educational Progress) A lead- 
ing spokesman for NAEP has told us that one "unit of reporting 
is the area" and, as I shall explain later, one of the stated pur- 
poses of NAEP requires for its fulfiUment that one such area 
unit of reporting be the state, and that NAEP achievement test 
results must be used to afford comparisons among the state 
school systems in this country. 

The two-part principle presupposes that any national 
achievement testing program whose results are used to compare 
public school systems would prove to be coercive in respect to the 
public schools, because it would inevitably force teachers to teach 
for the tests. NAEP promoters would say that this two-part prin- 
ciple does not pertain to the national achievement testing aspect 
of NAEP because probability sampling is to be used in NAEP, 
and they claim that the use of this statistical device will prevent 
their achievement tests from becoming coercive. In the words 
of one of these promoters, it is claimed that under conditions of 
probability sampUng " 'teaching for tests' and revising curricula 
to conform to test content would not be stimulated." 

Dr. Hand was formerly on the staff of the College of Education at the 
University of Illinois and has been a distinguished writer in the field of 
secondary education. 

This article is an adapted version of an address given at the AASA 
Annual Meeting on February 16. 1966. It is reprinted, by permission 
from the March 1966 issue of The Educational Forum. 
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tantamount to saying that Congress has a right to know what 
the results of national achievement testing are. "because it is 
putting money into it." 

5. The next logical step would be to make participation in 
NAEP a requirement for receiving federal aid. It is now legal for 
the Commissioner of Education to require such participation on 
the part of all state school systems which receive federal aid 
under either the 1963 Vocational Education Act or the 1965 
Elementary and Secondary Education Act. Both Acts make it 
obligatory upon the Commissioner to require that evaluations be 
submitted to him by all state school systems which receive federal 
aid under either Act, but neither Act stipulates what the required 
evaluation shall or shall not consist of. Instead, this is left to 
the judgment of the Commissioner. 

6. From here on out, virtually no state and but relatively 
few communities can support a quality public school program 
without federal aid. To say that states and public school districts 
any longer have the freedom to refuse federal aid is a cruel 
fiction. So virtually all of fhem would then be forced to go along 
with the national achievement testing program. 

But even if the national achievement testing aspect of 
NAEP continued to remain in the hands of a private gi'oup. with 
the test results used by the USOE for purposes of making a 
wiser — hence a differential — use of federal aid. the NAEP 
achievement tests would become coercive, and control of the 
public school curriculum would be in the hands of the private 
group who has the final say-so as to what goes into the NAEP 
achievement tests. 

We shall see why this is true as we take note of the seventh 
statement. 

7. Probability sampling cannot discourage, much less pre- 
vent, a great many teachers from teaching for the tests — quite 
the contrary. It is the use made of the test results that will be 
determinative. The NAEP promoters tell us that the NAEP 
assessment "seeks to describe ... by different geographic areas." 
This euphemism means that the public school systems are to be 
compared state by state. What is true of the sample is true of 
the whole; this is what probability sampling means. If the per- 
formance of the sample public schools in State X is reported to 
be low in comparison with the sample performances in other 
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test results were to be reported by larger regions rather than 
by states. If this is done, and if the stated purpose of NAEP of 
affording "a much more accurate guide to the allocation of . . . 
public funds'* is still operative, the USOE would still have to rank 
the stales on the basis of NAEP test results. But instead of there 
being only one state with Rank No. 1 there would then be two 
or more stales with Rank No. 1, two or more with Rank 2 and 
so on down the line. About half the rankings would be below 
the average, of course. So the image of all the public schools in 
all the states in all the regions with below-average rankings would 
be damaged. New industry would be frightened away from 
groups of states rather than frightened away on a state-by-state 
basis. The heat which the power structures in the threatened 
regions would turn on the public schools to "get those test scores 
up or else" would not be one whit less intense. Frustrated public 
school boards would then be joined in their misery by frustrated 
state boards of education. For each state board of education 
would be held accountable for the poor test results in all the 
states in its region. Under regional reporting, no state board of 
education could prove that its state deserved any higher rating 
than the rating for the region as a whole. 

All that I have said regarding the coercive consequences of 
NAEP achievement testing would still hold even if the USOE 
never has any official knowledge of the results or never makes any 
use of them. All that is required is that these results he known 
to the public. For once they are knov;n, the ranking by states 
and/or regions would follow. We can be sure that this ranking 
would be taken seriously, so great is the prestige of the sponsor- 
ing Carnegie Foundation, Tlie upshot would still be that the 
private parties who determine what goes into the tests would 
control the public school curriculum. 

Nor would the question of coercion depend upon the kind 
or kinds of questions which might be embodied in the NAEP 
achievement test items. For it is the resultant image of the 
pubhc school systems of a state or region, and not the kind or 
kinds of test items used, which would count with the power 
structures of the states or regions. 



We come now to the two-part principle I wish to propose. But 
before stating this principle, let us take note of three specitica- 
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such bodies the country over would be to the new public school 
policy which participation in NAEP would represent. Instead, 
and despite the fact that these bodies and they alone are legally 
empowered to establish new public school policies, the NAEP 
promoters consulted only with persons acting in their capacities 
as private citizens. True, some state and local school board 
members may have attended the invitational conferences, but 
if they did, they could speak there only as private citizens in 
respect to any new policy for the public schools. In sober fact, 
then, what the NAEP promoters did was to ignore completely the 
only bodies legally empowered to establish the new public school 
policy which these promoters are attempting to bring into being. 

Furthermore, the NAEP promoters did not make a sys- 
tematic factual study of and restrict themselves to ongoing public 
school practices set up under policies made by legally appointed 
or elected officials. To operate within the principle of consent 
the NAEP promoters would either (a) have to restrict their 
achievement test items to what is 100% common among the vast 
range of ongoing public school practices in this country or (b) 
ask each public school caught in their probability sample to delete 
from the NAEP tests all items not appropriate for that particular 
public school. Yet the NAEP promoters didn't even draw a prob- 
ability sample of public school systems to get official reports 
indicative of what is included in even a representative sample of 
the universe of ongoing public school practices, to say nothing 
of making a study to see which if any of the ongoing practices 
within this sample of the said universe were everywhere manifest. 

Instead, the promoters of NAEP proceeded at once to award 
contracts to private testing agencies for the development of 
achievement tests to be used in public schools. 

Every achievement test item presupposes the desirability of 
the educational objective to which the item relates. So the pri- 
vate testing agencies had to start by setting down what they think 
the educational objectives of the public schools should be, and 
to go on from there. The private commission in charge of NAEP 
is to have the final f=ay-so in respect to the tests. But they are in 
the same boat and so must pass judgment on the tests in terms 
of what they believe the educational objectives of the public 
schools should be. A private body, the Carnegie Foundation, is 
footing the bill. 

Private conferees, private test makers, private test approv- 
ers, and a private financial angel — there is not a person among 
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objectives, the subject matter content used, the teaching methods 
employed, and the way pupils are treated — all this to whatever 
extent it is intended that this, that, or the other aspect of the total 
program of any sample school system is to be assessed by part or 
all of the NAEP assessment battery. 

There are three answers to this objection. One is that if any 
such external agency as the private commission in charge of 
NAEP cannot secure this information, then this agency could 
not possibly know what would constitute technically valid meas- 
ures of achievement for the schools in question. 

The second answer is that if the private commission cannot 
secure the information, then it is virtually certain that it could 
not possibly conduct the appraisal either within the apparatus 
of state and local control of the public school curriculum or 
within the apparatus of government by consent— unless, of 
course, the appropriate NAEP tests are to accompany ever^ 
invitation to participate, and unless the stated expectation is 
that all the NAEP tests are to be modified by state and local 
officials to make them conform to legally sanctioned state and 
local conditions. 

The third answer is that this information is known, but 
for the most part only to the school people within any given 
public school system itself. Some of the most important parts 
of this information relate to intangibles or quasi-intangibles 
which are sensed but seldom a matter of written record, e.g., 
the concepts of self-worth and self-confidence on the part of 
pupils the teachers are trying to help develop, the human rela- 
tionships among the youngsters the school is trying to engender, 
the kinds of viciously miseducative neighborhood environments 
the staff is trying to compensate for, and the appreciations and 
aspirations the teachers are hopeful of instilling in the children. 
But whether a matter of written record or merely sensed, a sig- 
nificant amount of this information can be communicated only 
partially to any outside group. What is required, then, is that all 
tests and other instruments by which public school systems are 
to be assessed must be shown to and their validity judged by the 
teachers, counselors, supervisors, and administrators of the 
various school systems if a partial, warped, and unfair judgment 
of the public schools is to be avoided. 

I am aware that something of this sort presumably is or will 
be going on in the few schools in which the NAEP instruments 
are now being or will be tried out, but we are unbelievably naive 
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The Assessment 
Debate at the White 
House Conference 

A lively clash of ideas among leading educa- 

u y 20 71965?' "7" - Educafion t 

July 20 [1965] when Panel 2-B ("Assessment of Educational 
Performance- ) chose to discuss the question of natiLal a sess. 
ment now being studied by the Carnegie Corporation 
a proDOsa ^°"^"^'«"VJ°hn Goodlad opened the discussion with 
a proposal that a nationaUy televised colloquy accompany the 
begmnmgs of assessment to insure a general undersSng of 
the twin problems of clarifying educational goals and oFassess- 
mg their attainment. assess- 

tlon?^ll°HLV^''' assessments of American educa- 

tion/ A discussion summary follows. 

CARR: Yes and this is being done every day by those who 
should continue to do it. the teachers in^ the classroom We 
should not substitute a national assessment for this. Pluralism 
m testing should be retained. Assessment should be used S 
help teachers do a better job. 

DUNNAN: Ingenuity, flexibility, and initiative can be lost in a 
national testing program. [Note the shift from the quest' oi of 
assessment to "a national testing program."] We have two 
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Reprinted from the September 1965 issue of the Phi Delta Kamynr, 
Participants in the panel discussion were: Chaiman HarolT B 
Gores president. Educational Facilities Laboratories T^c New V„rS' 

SnSr-o/ci' UniversrEima^'"scI°c2 
i;«n Of CaWomia at Los Angeles; Questioner: Stephen K Bailev 
dean. MaxweU Graduate School of Citizenship and Public AffaSs Syracuse 
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i>"cw lorK jack Arbolino. director. Advanced Placement Proeram rnii« J 

Entrance Examination Board. New York; Thomas W BraS^^^'re^S 

California State Board of Education- Wiin,™ r President. 
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of schools. St. Paul. Minnesota ' *"P«""'«"dent 
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BRADEN: Overcoming prejudice, establishing a value system, 
and developing factual habits of mind are the real goals we must 
learn how to assess. 

DONOVAN: Assessment is going on all the time, not only by 
teachers but by parents, voters, employers, legislators, and others. 
I question our ability to construct any valid system of central 
assessment. The divisive effect of any effort to quantify our 
strengths and weaknesses may not be worth the gain. 

BAILEY: And if our goals are based simply on non-prejudice, 
a set value system, and pure rationality, how do we create the 
warmth, the uniqueness, and the unpredictability so important 
in our society? Our discourses must produce warmth as well 
as light. 

DUNNAN: We must be most concerned with understanding 
the minds of our students. Man has lived on this earth three- 
quarters of a million years and has only begun to learn the rudi- 
ments of the way the human mind works. The future is limitless. 

GOODLAD: And we have only begun to utilize the potentials of 
assessment. Today we are happy if a fifth grader with an IQ 
score of 100 reads at the sixth-grade level, unconcerned with 
where he should be reading, based on his ability, not the group 
average of attainments. All fifth graders can be reading at the 
fifth-grade level and all be deficient in some aspect of reading 
going undiagnosed. We have done almost no assessment based 
on criterion measures rather than norm-based standards. The 
difference between assessing individual students and assessing 
our total educational effort with its resulting educational per- 
formance is enormous. 

We have begun to be able to evaluate such competencies 
as inquiry and decision making, but we use these assessments 
hardly at all in determining the adequacy of our educational 
effort. Especially in teacher education we still assess chiefly 
knowledge of facts about schools, about child psychology, etc. 
We must assess various kinds of talents suitable for different 
kinds of teaching tasks and report the kinds of teaching abilities 
being developed. We have three distinct jobs: identifying de- 
sirable pupil traits; learning to assess them; and deciding which 
ones to value. 

ARBOLINO: Can we assess student achievement without en- 
couraging conformity? 
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LINDLEY STILES (dean, School of Education, University of 
Wisconsin): What are we afraid of? We compare people on 
physical fitness, color blindness, mental retardation, and other 
traits. Why not compare them on academic achievement? The 
NEA assesses states on their treatment of teachers and applies 
sanctions to those which fail to measure up. 

JOHN H. FISCHER (president, Teachers College, Columbia): 
What objection is there to an assessment that seeks to collect 
data on the condition and progress of education in the several 
states? The only objections we have heard are hints that if we 
get more significant information someone might be tempted 
to misuse it. Isn't that somewhat irrelevant? 

JERROLD ZACHARIAS (Massachusetts Institute of Technol- 
ogy): Fact-finding we need; new tests we need; testing the tests 
we need. But we do not need to hang a sword of Damocles over 
the head of the child by instituting ruinous pressure of com- 
petition. 

HAROLD TAYLOR (former president, Sarah Lawrence College) : 
We need not more tests but fewer interruptions of learning. At 
Sarah Lawrence or anywhere else, when you abolish tests you 
get more learning. Europe, the Middle East, and the Far East 
illustrate the paralysis of education produced by standard tests 
linked with government assessment. Let's abolish all tests and 
concentrate on teaching and learning. 

SISTER JACQUELINE (president, Webster College): Any unit 
—town, state, or nation— that employs a monolithic set of 
standards is stifling growth. It doesn't have to be a national test. 
GOODLAD: The comments about needing more knowledge and 
needing to keep the pressure off the students emphasize the 
difference between fact-finding and testing. A national assess- 
ment can provide information without setting standards or 
creating competitive pressures. We do need to know what 
students know and how they think. And we can discover this 
without creating pressures. 

For one thing, we can select various samples of children 
from different localities, administer to each section of the sam- 
ple 10 percent of the total test. Assembling the results, we can 
achieve the desired knowledge of student attainment in many 
aspects of life without ever pressuring or punishing or grading 
children, but simply finding facts. 
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RESOLUTION OF _ _ 

rmr?cTA?^ Measurement and 

CHIEF STATE _ 

SCHOOL Reporting of 

OFFICERS — I , ^ 

NOVEMBER 1965 Educational Besults 



We recognize that expansion in quality, scope 
and cost of education brings new responsibilities to the educa- 
tional agencies of local, state and federal governments to evaluate 
public education and to report the results of these evaluations 
appropriately to pupils, parents, teachers and the general public 
The people have a right to know about the results of education 
and Its cost as the largest peacetime function of government The 
action of the Congress in making evaluations of educational pro- 
grams at the state and local levels part of the requirements for 
the use of supporting federal funds reflects the general public 
desire for educational accounting. Despite the difficulties in- 
volved we believe public school administrators have an obligation 
to make such an accounting. In the process, it should be possi- 
ble to assist both the profession and the public to understand 
more thoroughly that the complexities of teaching and learning 
hmit the extent to which the results of education can be meas 
ured with precision. 

We believe local and state educational agencies can and 

S in^'J^'fi '"P"^^' °^ "^^"'"'"g '""^^^'^d responsi- 

bility in this field. In our times, most important areas o.f human 
endeavor are organized and conducted with reference fo local 
state, national and even international environments. Amcul' 
ture, transportation, communication, commerce, finance labor 
science, research and social trends of all kinds operate in in- 
creasingly broader environments, and so does education We 

imited objectives and methods affecting teaching and pupil 
learning m schools should reflect these realities. The signS- 
cance of the widening environment within which education 
operates is clear. Both the scope of the curriculum and he 
fnTo account achievements in schools should take it 
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liance on their mathLaUcallv ivn \ ^'^^ '"e- 
careful restriction on t^ use of '^'i. ^^'^ ^''•ll 

Which they cleariy serve Ts h^trds^'S'"'^ '° P"^"^'^^ 
striction of the use or nnr „ i 'ead us to favor re- 

tion and clinical work by~ '^^f" -P-^-L 

about IQ and other tesL °^ '^'"S'"g ^i'^ws 

bringing general recog^ tion Z 'env'^'"'" ^" '''''' y^^^' 
part in the developmS of ml P'^^' ^ g^'^^ter 
been previously supS L intelligence than had 

upon ability test scores in 1 "1^ ^'"""'^'^ 
abuses of these scores P'^^^ment of pupils and other 

reliable and valid of all tTsts and thTr^.^ ""^J^^^^^' 

erally beneficial to pupils fache" , LnH '^'^ "^^^ "'^^^ g^"" 
The hazards connected wUh ' dev.In"'"'^"""' 

be recognized and can be contrS '^fJ^^'^Pj?'^"' use must 

such testing instruments anS s theiTres^^^^ '^^^'^'"P 
local, state and federal authorise 1 ° cooperate with 

with local classroom iLtructr n . '"'"^""^ interference 
primarily to raise test sco™and ?n i!'^ 
scores. Parents, school sTa^ .n "''''"^ '"^'"^^^ 
members, leaders in the mas ' m^'"''''f ''^"^^ ^oard 
citizens should work together to Zltl '"'^ '"^^"igent private 
'ogetner to achieve general understanding 
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of desirable uses of appropriate instruments for the measure- 
ment of academic achievement. 

We recognize the need for improvement in the assessment 
of American education. Such assessment should be developed, 
administered and interpreted, however, so as to take into full 
account the wide diversity in the social, economic and ethnic 
character of our nation and in the aspirations of our people. 
Agreements among states providing for interstate assessments 
of educational results can supplement local and state evalua- 
tions and further local, state, and national cooperation in edu- 
cation in terms of local, state and national needs. It is essential, 
therefore, that any assessment should be prepared in coopera- 
tion with and administered by the states and localities. 
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execJtTve <*«idelines for National 
smtS? Assessment of 
JANUARy 1966 Educational Outcomes 

rp 

for Supervision and' Cu^Tcu\L'' nTl"'' °' Association 
effort to keep the membership rfo^ ^°f"?'^"' "^^^e an 

area of national assessment of "du^M "["^'"^^^^"'^^^^ i" the 
''on to reporting developln s and T'"""''' ^" 

•ssues, the Committee is '"'^''^a^'ng problems and 

that Should be evident in an^ o^^^^^^^^ ^^--ns 
mented. ' PT^o]ect that is proposed or imple- 

for3ouL"XrraTd"Xtiver"f 

educational structure Ass2ment LT f'°" ^'"""'^ 

aspect of curriculum deve'pmem and is '"'^^^ral 

of curriculum workers. This^espons.hi '"esponsibility 

m a time of awakened publ c conco^^ especially critical 

ment and widespread professional ST' ""'"V" ^"""^^'^^ 

endeavors. It is' there ore ^c ;^^^^^^^^^^ 

everywhere develop new p^ocedTrPc f eumculum workers 

present levels to meet p^operlvlr . ."'^'^^^'^^nt far beyond 

Whatever goals orpfoced^rL fo^^ ""'^^ 
to education have inevitable efft rupo^ ^PP'^^d 
of the curriculum. This inescambIoT.Tr T""'^ ""'^ ^""etion 
understood by curriculum nnH . ' '°"'^'P "'"^^ be clearly 
consequences'of bhndneTs "r wf " ^'^'^ 

destroy with one hand what is bu l/ n? ^^^-^d 'o 

Any assessment program r^u the Ifor^''' ^''^ °'her. 
ing criteria: '^'''^''f°'^e meet at least the follow- 

-^^/eo,f"?h:Unit:dtaS^^^^^ 

its proudest tradition rest Tpo^V,^^"" °' "'^"^ cultures Ind 
-dividual and the protect n of "h r £ ' ""^'^ 
en^ssment must value Jll^f diS^^^f s'^ch^Si: 
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properly expressive of the ethnic, religious and geographic back- 
grounds of our people and fulfilling the multiplicity of objectives 
demanded by local communities of their schools. Sensitivity of 
schools to local needs is an essential characteristic of a demo- 
cratic society. 

2. Assessment must protect and encourage uniqueness in 
students and citizens. Even as plurality of cultures characterizes 
our society so opportunities for individual development and 
growth must be prized. This includes an awareness of the cul- 
tural heritage of the individual and a chance to go beyond that 
heritage in terms of the individual's own growth and potential. 
A dynamic society cannot afford to stifle creativity and unique- 
ness in its young. 

3. Accurate assessment of educational outcomes requires 
exploration in breadth across the full range of educational ob- 
jectives. This includes social, emotional, vocational, health and 
artistic goals as well as basic skills and intellectual growth 
Where valid procedures for assessment of these broad objectives 
do not exist, they must be developed. Data obtained from a 
hmited sample of these objectives or from limited instruments 
can provide but a distorted picture of educational accomplish- 
ment. 

4. Adequate assessment also requires exploration of learn- 
ing in depth. Learning may vary from superficial "knowing" to 
effective, efficient "behaving." It is not enough that schools 
produce students who "know" better. The only valid criterion 
for effective learning is whether the student behaves differently 
as a consequence of having participated in the process. Proper 
assessment must be directed to the deeper questions of effective 
behavior. 

5. Assessment must explore the changes in students over 
periods of time. The static assessment of a sample group of 
learners at a particular point in time fails to distinguish the 
effects of education from the effects of parental or other en- 
vironmental influences. Adequate assessment programs must 
include measures of growth and change to assess properly out- 
comes. ^ 

6. The effect of the assessmejit procedures themselves on 
the nature aiid function of the educational process must be ap- 



p^ccmtcd and continuously subject to scrutiny. As larce-scale 
assessment of the effectiveness of education begins to take place 
here are dangers that eurrieula will be tailored to f t SaLa Ln 

Srsurxhrn ^'"'^ ""'^"^"^'^y replace t 1 ; 

u e of l.n large-scale assessment is the 

use of data as the basis for further dialog and clarific.ion nr 
the objectives and means of education. It'musttr h rather 
than inhibit the above stated objectives. 
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RESOLUTION 
FEBRUARY 1966 CUmCUlum 

ic.ra, "^^^ American Association of School Admin- 

ist ators opposes any act which would, in effect es abhsT a 
national testing system or a national curriculum No onlv 
does such a high degree of centralization infringe upon^he S 
esponsibilities of the state and the school system and the pro 
fessional responsibilities of the individual teacher but wo Hp 

tmn- he individual development of each child. Attempts to 

cstw%7esurir^" '''r' ^^^^^^^ prescribe^nTn 
tests will result in a curricular structure which will vitiate if 
tempts of local schools to serve individual pupils 

The AASA believes that the voluntary cooperation of loc-il 
school districts, education associations, foundation state Se 
partments of education, and regional agencies ^^,1 
producing sufficient data for reseich, evaluation, or asses men 
pu poses and as much standardization of curriculum as con 
sistent with the maximum quality desired. 
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and National 
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